A Statistical and Rule-based Method for Chunking Verbal Units in Thai Texts

Tokenizing a text into a sequence of words is an important process towards text interpretation. This process is required in many applications such as text summarization, semantic search, and machine translation. Instead of splitting into words, recently there have been works on chunking into units which are larger than words. Text chunking is a process to divide a running text into non-overlapping groups of words, which have meaningful contents, such as named entities and verbal units. In this work, we explore three layers of verbal units, called (1) verbal sequences, (2) verb phrases (i.e., verbal chunks, causative forms and event occurrences), and (3) elementary discourse units (EDUs). As the basic layer, a verbal sequence is defined as a single verb or a sequence of contiguous verbs without any interrupting nouns or particles. For example, A Statistical and Rule-based Method for Chunking Verbal Units in Thai Texts